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Abstract. OpenFMO framework, an open-source software (OSS) platform for Fragment Molecular Orbital (FMO) method, 
is extended to multi-physics simulations (MPS). After reviewing the several FMO implementations on distributed computer 
environments, the subsequent development planning corresponding to MPS is presented. It is discussed which should be 
selected as a scientific software, lightweight and reconfigurable form or large and self-contained form. 
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INTRODUCTION AND OVERVIEW 

Multi-physics simulations are widely used even in complex scientific studies. Such calculations are often constructed 
by combining multiple theories including different degrees of approximations and different scales of description. 
Since reality and accuracy are required increasingly, these simulations have become larger and more complicated year 
by year. Grids, distributed computer resources over wide-area networks, are expected to execute such complicated 
scientific applications, and have been installed all over the world in order to demonstrate large-scale heterogeneous 
simulations with the help of middlewares 111 0]. On the other hand, the next generation supercomputer with a peta- 
scale performance is already planned in several countries iSH]. Thus, the development of high-performance computing 
environments is fast and transient. As a scientist, it is important to watch the trend of those computer resources. 

In the present contribution, the multi-physics calculations by Fragment Molecular Orbital (FMO) method [5] are 
constructed on the distributed computing environments. OpenFMO framework toward "peta- scale" computing^ [tI] 
is extended to the multi-physics simulations. It is also discussed what architecture and development policy should be 
chosen in the fast-moving world of computing. 

GRID-ENABLED CALCULATIONS OF FMO 

Before entering the main subject, we briefly review the grid-enabled FMO implementations developed in the NAREGI 
projectLU. These are based on the famous MO package, GAMESS|0]. 



Implementation of a Loosely-coupled FMO 

Although it is usually considered as an approximation to ab initio molecular orbital (MO) calculations, the FMO 
algorithm is a multi-layered problem (see Fig.[TJb)) including the MO calculations for each fragment and the electro- 
static (ES) interaction between fragments. In the MO-layer, the quantum mechanical interactions of all the atoms 
and electrons within a fragment are included to obtain a fragment energy. On the other hand, only the classical ES 
interaction is considered when we go over the boundary of fragments. Since the MO-layer calculations can be executed 
independently, we can break the program into loosely-coupled components corresponding to a large-scale parallel 
execution in the distributed computing environments (Fig.[TJc)). 
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FIGURE 1. The structure of Loosely-coupled FMO represented in (a) a flow chart, (b) a program stack, and (c) graphical icons 
on the NAREGI Workflow tool. 



FIGURE 2. The electron density obtained by FMO is shown by the use of the NAREGI visualization system: (a) the total electron 
density of a Gramicidin- A (UNO); (b) the electron density of a fragment in a fatty acid- albumin. 



The grid-enabled version called "Loosely-coupled FMO" was developed as a part of NAREGIf 1]. The total control 
flow is constructed by the use of the NAREGI Workflow tool. In Fig. [2] (a), the total electron density of the whole 
molecule [9] of a Gramicidin- A is shown as an equi-density surface, and the electron density for one of the fragments 
in a fatty-acid albumin is shown in Fig. Ob). 



3D-RISM/FMO Simulation Connected by a Mediator 

As an example of the multi-physics simulations, a coupled simulation of FMO and 3D-RISM is presented, where 
FMO calculations are coupled to statistical mechanics calculations for molecular liquids by Reference Interaction Site 
Model (RISM)|10]. In order to obtain properties of bio-molecules, drugs, enzymes, etc., it is necessary to perform 
calculations under the influence of a solvent since these molecules usually work in aqueous solution. However, the full 
description of the solute and solvent system is difficult in general because of the large number of degrees of freedom. 
The standard strategy to solve the problem is to combine, in some way, originally different theories or programs, which 
is the multi-physics approach. 

In the multi-physics simulations, physical data are exchanged between separate program components, where we 
must transform not only formats but also their semantics, i.e., physical meanings of the data. In order to assist such data- 
exchanges with semantic transformations, we used a set of application program interfaces called Mediator (mediator- 
API) [fill, T], which is included in the beta- version release of the NAREGI grid-middleware. Fig.[3ta) shows the total 
flow of this simulation, where the partial charge distribution of the solute and solvent molecules are exchanged each 
other through the mediator- API (Fig.[3tb)). In order to execute on the NAREGI grid, the flow is incorporated in the 
NAREGI Workflow tool (Fig.[lc)). 

In Fig. m we show results of this coupled calculation for methionine-enkephalin (75 atoms) and chignolin (138 
atoms) in aqueous solution, where the partial charge distribution by water molecules are also shown around these 
molecules. 
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FIGURE 3. The structure of 3D-RISM/FM0 represented in (a) a flow chart, (b) a program stack, and (c) graphical icons on the 
NAREGI Workflow tool. 




DEVELOPMENT OF OPENFMO FRAMEWORK 

OpenFMO|6] is an open-licensed software platform to construct FMO applications under high-performance distributed 
computer environments. The current status of this development is in the end of Phase II. In Phase I, we introduced the 
OpenFMO framework and predicted a peta-scale performance on a hypothetical computer architecture 0]. In Phase II, 
we have tried to implement the skeleton by one-sided communications! 12] under the PSI project^?]. In Phase III, we 
are going to extend the platform to the multi-physics simulations (see Fig.O. 



Multi-physics Extension of OpenFMO and its Application 

The main purpose of the Phase II in the development schedule of OpenFMO (Left of Fig. [5]) was to correspond 
actual executions on the next generation supercomputer with more than 10,000 CPUs, where we have tried to reduce 
redundant memory consumption on each computing nodes and improve parallel performance by the use of one-sided 
communications. The detailed results will be presented elsewhere 1 12]. 

The OpenFMO framework is extended to correspond to multi-physics applications including scientific simulations 
(Fig. O. Since the current FMO skeleton is well configured and has been proven effective in high-performance 
computing environments, it is better that the other multi-physics components are developed separately. Then, the 
key point is physical data representation and manipulation, where sort of transparent accessibility to internal data from 
outer components should be provided. The semantic transformation of the data can also be executed by the Mediator 
component depending on the needs. 

By the use of the multi-physics extension, one can construct the following coupled simulations: QM/MM or 
QM/MD calculations! 13, 14] for docking simulations of proteins and enzymes in aqueous solution, RISM/SCF 
simulations! 10] for various protein molecules with scientific theoretical studies! 15], etc. One of the properties of the 
OpenFMO platform is the lightweight and reconfigurable skeleton program, which is useful for modifying applications 
corresponding to fast-changing computational environments. 



Phase I (2006) 

o Fixing interfaces of MO library programs 
o Skeleton with GAMESS-type algorithm 
o Performance prediction for peta-computing 

Phase II (the first half of 2007) 

o New skeleton for high-performance computing 
o Performance analysis using MPI-2 functions 

Phase III (the latter half of 2007) 
o Multi-physics extension 
o Scientific applications 



FIGURE 5. Left: Development planning of OpenFMO. Right: Stack structure of OpenFMO with multi-physics extension. 



DISCUSSION 

Application sizes of recent molecular package programs are more and more increasing by their self-contained struc- 
tures with many functions corresponding to complicated options and various computer environments. Such strategies 
in their development potentially run into a dead-end. However, when we use multiple grid resources simultaneously, it 
is inevitable to prepare a tailored schedulerfl^ in each application. The most important task as a scientist is to develop 
effective theories and algorithms that can be implemented on every computer environments, while actual implemen- 
tation on a given environment should be carefully done with the help of computer scientists. Our development of the 
OpenFMO framework is one of those attempts to implement the FMO algorithm on the future computers. 

This work is partly supported by the Ministry of Education, Sports, Culture, Science and Technology (MEXT) 
through the Science-grid NAREGI Program under the Development and Application of Advanced High-performance 
Supercomputer Project. 
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